Building A Tool For Annotating Reference In Discourse

We discuss the development of a system for marking several types of reference to facilitate the analysis of reference in discourse. The tool is designed to be used in three applications: generating training data for machine learning of co-reference relations, evaluating theories of referring expression generation and resolution in texts, and developing theories for understanding reference in dialogs. The need to mark any of a broad set of relations which may span several levels of discourse structure drives the system architecture. The system has the ability to collect statistics over encoded relations and measure inter-coder reliability, and includes tools to increase the accuracy of the user’s markings by highlighting the discrepancies between two sets of markings. Using parsed corpora as the input further reduces the human workload and increases reliability.